The present invention relates to an image signal data structure, an image coding method, and an image decoding method. More particularly, the invention relates to an image signal data structure which includes reproduction timing data relating to the timing of reproduction including decoding and image display for each of frames as components of an image, generation (coding) of a coded image signal including the reproduction timing data, and decoding of the coded image signal.
Further, the present invention relates to an image coding apparatus for generating the coded image signal, an image decoding apparatus performing the above-described decoding, a data storage medium containing a coded image signal of the above-described data structure, and a data storage medium containing an image processing program for implementing the above-described coding and decoding by using a computer.
In recent years, we have greeted the age of multimedia in which audio, video and other data are integrally handled, and the conventional information media, i.e., means for transmitting information to men such as newspapers, magazines, televisions, radios, and telephones, have been adopted as the objects of multimedia. Generally, xe2x80x9cmultimediaxe2x80x9d means to represent, not only characters, but also diagrams, speeches, and especially images simultaneously in relation with each other. In order to handle the conventional information media as the objects of multimedia, it is necessary to convert the information of the media into a digital format.
When the data quantity of each information medium described above is estimated as a quantity of digital data, in the case of characters, the data quantity for each character is 1xcx9c2 byte. However, in the case of speech, the data quantity is 64 kbits per second (quality for telecommunication) and, in the case of moving picture, it is more than 100 Mbits per second (quality for current television broadcasting). Thus, in the information media such as televisions, it is not practical to process such massive data as it is in a digital format. For example, although visual telephones have already been put to practical use by ISDN (Integrated Services Digital Network) having a transmission rate of 64 kbpsxcx9c1.5 Mbps, it is impossible to transmit an image of a television camera as it is by the ISDN.
As a result, data compression technologies are demanded. In the case of visual telephones, a moving picture compression technology standardized as H.261 by ITU-T (International Telecommunication Union-Telecommunication Sector) is employed. Further, according to a data compression technology of MPEG1, it is possible to record image data, together with audio data, in an ordinary music CD (compact disk).
MPEG (Moving Picture Experts Group) is an international standard relating to a technology for compressing and expanding an image signal corresponding to a moving picture, and MPEG1 is a standard for compressing moving picture data to 1.5 Mbps, i.e., compressing data of a television signal to about 1/100. Since the transmission rate to which MPEG1 is directed is limited to about 1.5 Mbps, MPEG2 capable of compressing moving picture data to 2xcx9c15 Mbps has been standardized to meet the demand for higher image quality.
In the image signal compression and expansiochnologies according to MPEG1 and MPEG2 which have already been put to practical use, only a fixed frame rate is basically employed, namely, intervals between image display timings of the respective frames are regular. As a result, there are only several kinds of frame rates, and in MPEG2 a frame rate designated by a flag (frame rate code) which is transmitted with coded data is selected from plural frame rates (frame rate values) with reference to a table shown in FIG. 13.
Under the existing circumstances, standardization of MPEG4 is now proceeded by the working group for standardization of MPEG1 and MPEG2 (ISO/IEC JTC1/SC29/WG11). MPEG4 enables coding and signal operation in object units, and realizes new functions required in the age of multimedia. MPEG4 enables coding and signal operation in object units, and realizes new functions required in the age of multimedia. MPEG4 has originally aimed at standardization of image processing at a low bit rate, but the object of the standardization is now extended to more versatile image processing including high-bit-rate image processing adaptable to an interlaced image.
Also in MPEG4, when a table similar to the table for MPEG2 (refer to FIG. 13) is added at the beginning of a video object layer (corresponding to a video sequence of MPEG2), frame rates can be expressed according to the table. In MPEG4, however, since image signals in a broad range from an image signal of a low bit rate to a high-quality image signal of a high bit rate are processed, the number of frame rates required is out of count. Therefore, it is difficult to perform decision of frame rates by the use of a table.
As a result, MPEG4 employs a data structure including frame display time data inserted in each frame in order to deal with almost uncountable number of fixed frame rates and, furthermore, to process an image having variable intervals of image display timings or decoding timings of the respective frames.
FIG. 14 shows a data structure of a conventional coded image signal 200.
The coded image signal 200 corresponds to one image (in MPEG4, a series of frames constituting an image corresponding to one object) and includes a header H at the beginning. The header H is followed by code sequences Sa0, Sa1, Sa2, . . . , San corresponding to frames F(0), F(1), F(2), . . . , F(n), respectively, which code sequences are arranged according to priority for transmission (transmission order). Here, xe2x80x9cnxe2x80x9d is the number indicating the transmission order of data of each frame in the frame sequence corresponding to one image, and n of the head frame is 0.
In this example, at the beginnings of the code sequences Sa0, Sa1, Sa2, . . . , San of the respective frames, display time data Dt0, Dt1, Dt2, . . . , Dtn indicating the display timings of the frames are arranged. The respective display time data are followed by coded image data Cg0, Cg1, Cg2, . . . , Cgn.
Since each of the display time data indicates a time relative to a reference time, the quantity of data required for expressing this display time, i.e., the bit number of the display time data, increases as the number of the frames constituting the image increases.
Further, at the decoding end of the coded image signal, according to the display time data Dt0xcx9cDtn inserted in the code sequences Sa0xcx9cSan corresponding to the respective frames, image display of each frame is carried out at the time indicated by the display time data.
FIG. 15 shows the transmission order and the display order of the coded image data corresponding to each frame in the series of frames. As described above, xe2x80x9cnxe2x80x9d indicates the transmission order, and xe2x80x9cnxe2x80x2xe2x80x9d, indicates the display order (nxe2x80x2 of the head frame is 0). Further, frames F(n) (F(0)xcx9cF(18)) are arranged based on the order of frames in the data structure shown in FIG. 14 (transmission order). The frames F(n) arranged in the transmission order are rearranged according to the display order of the frames as shown by arrows in FIG. 15, resulting in frames Fxe2x80x2(nxe2x80x2) (Fxe2x80x2(0)xcx9cFxe2x80x2(18)) arranged in the display order. Accordingly, a frame F(n) and a frame Fxe2x80x2(nxe2x80x2) related to each other with an arrow are identical. For example, the frames F(0), F(1), F(2), and F(3) are identical to the frames Fxe2x80x2(0), Fxe2x80x2(3), Fxe2x80x2(1), and Fxe2x80x2(2), respectively.
Amongst the frames F(n) (F(0)xcx9cF(18)) arranged in the transmission order, the frames F(0) and F(13) are I (Intra-picture) frames (hereinafter also referred to as I-VOP), the frames F(1), F(4), F(7), F(10), and F(16) are P (Predictive-picture) frames (hereinafter also referred to as P-VOP), and the frames F(2), F(3), F(5), F(6), F(8), F(9), F(11), F(12), F(14), F(15), F(17), and F(18) are B (Bidirectionally predictive picture) frames (hereinafter also referred to as B-VOP).
When the frames F(n) (F(0)xcx9cF(18)) arranged in the transmission order (IPBBPBBPBBPBBIBBPBB) are rearranged in the display order (IBBPBBPBBPBBPBBIBBP), this display order nxe2x80x2 is represented by frame numbers B(n) (B(0)xcx9cB(18)) corresponding to the respective frames F(n). That is, the frame numbers B(n) represent the numbers nxe2x80x2 indicating the display order. To be specific, as shown in FIG. 15, B(0)=0, B(1)=3, . . . , B(17)=16, B(18)=17. Accordingly, the image display cycle L of the I-VOPs is 15, and the image display cycle M of the VOPs including both of the I-VOPs and the P-VOPs is 3.
The frame number B(n)=nxe2x80x2 is represented by the following formulae (1)xcx9c(3) using n.
B(n)=n=0(n=0)xe2x80x83xe2x80x83(1)
B(n)=n+Mxe2x88x921(n=Mxc3x97i+1)xe2x80x83xe2x80x83(2)
wherein i and M are integers not less than 0 (0, 1, 2, . . . ).
B(n)=nxe2x88x921(when n is other than the above values)xe2x80x83xe2x80x83(3)
The first I-VOP satisfies the condition (n=0), the I-VOPs other than the first I-VOP and the P-VOPs satisfy the condition (n=M=i+1), and the B-VOPs satisfy the condition (when n is other than the above values).
Formulae (1)xcx9c(3) define the relationship B(n)=nxe2x80x2 between the display order nxe2x80x2 and the transmission order n in the case where the code sequences of the frames corresponding to the respective I-VOPs, P-VOPs, and B-VOPs are transmitted periodically. In other cases than mentioned above, the display order nxe2x80x2 and the transmission order n are correlated one to one by a relational expression or a method other than formulae (1)xcx9c(3).
FIG. 16 is a diagram for explaining an example of an image display method wherein the intervals of the image display timings of the respective frames are variable.
In the FIG., txe2x80x2(nxe2x80x2) (txe2x80x2(1), txe2x80x2(2), txe2x80x2(3), txe2x80x2(4), . . . ) indicates the interval between the time at which image display of the frame Fxe2x80x2(nxe2x80x2xe2x88x921) is performed and the time at which image display of the frame Fxe2x80x2(nxe2x80x2) is performed, and hxe2x80x2(1), hxe2x80x2(2), and hxe2x80x2(3) indicate the times for image display of the frames Fxe2x80x2(1), Fxe2x80x2(2), and Fxe2x80x2(3), respectively, with the time hxe2x80x2(0) for image display of the frame Fxe2x80x2(0) as a reference. Further, h(n) (h(1), h(2), h(3), h(4), . . . ) indicates the time for image display of the frame F(n) (F(1), F(2), F(3), F(4), . . . ) with the time hxe2x80x2(0) for image display of the frame F(0)=Fxe2x80x2(0) as a reference. Accordingly, the display time hxe2x80x2(nxe2x80x2) of the frame Fxe2x80x2(nxe2x80x2) arranged in the display order is expressed by hxe2x80x2(nxe2x80x2)=hxe2x80x2(nxe2x80x2xe2x88x921)+txe2x80x2(nxe2x80x2), and hxe2x80x2(0)=0.
Next, decoding and image display of the coded image signal having the data structure shown in FIG. 14 will be briefly described using FIG. 16.
At the decoding end, when the coded image signal 200 shown in FIG. 14 is input, the coded image data Cg0, Cg1, Cg2, . . . of the respective frames F(0), F(1), F(2), . . . as the constituents of the coded image signal 200 are decoded, and the images corresponding to the frames F(0), F(1), F(2), . . . are displayed at the image display times h(0), h(1), h(2), . . . based on the display time data Dt0, Dt1, Dt2, . . . of the respective frames.
In this way, even when the intervals between the image display timings of the respective frames (image display cycle) of the coded image signal are not fixed, i.e., are variable, the coded image signal is decoded at the decoding end and displayed at a prescribed timing.
When the intervals between the image display timings of the respective frames of the coded image signal are fixed, as in the case where the intervals are variable, the images corresponding to the frames F(0), F(1), F(2), . . . are displayed at the image display times h(0), h(1), h(2), . . . based on the display time data Dt0, Dt1, Dt2, of the respective frames.
By the way, when expressing a frame rate (number of frames displayed in a second) simply with k bits (k: natural number), a frequency used for television broadcasting, for example, 29.97 . . . Hz (to be exact, 30000/1001 Hz) cannot be expressed.
As a result, such a frame rate is expressed as follows. That is, a prescribed time interval (1 modulo time), for example, one second, is divided into N (N: natural number) to obtain a sub-unit time (1/N) and, using this as a unit of time (1 time tick), the display time of each frame is expressed for both of the image having a variable frame rate and the image having a fixed frame rate.
To be specific, as shown in FIG. 17(a), the display time of each of the images VOP0, VOP1, VOP2, and VOP3 corresponding to the frames Fxe2x80x2(0), Fxe2x80x2(1), Fxe2x80x2(2), and Fxe2x80x2(3) arranged in the display order is expressed by y (VOP rate increment) pieces of 1/N (sub-unit time) with a time X as a reference, that is, it is expressed by y/N. For the images VOP1, VOP2, VOP3, and VOP4, y is defined as follows: y=yxe2x80x20, y=yxe2x80x21, y=yxe2x80x22, and y=yxe2x80x23, respectively.
FIG. 17(c) shows a coded image signal 200a having a data structure in which the image display timings of the respective frames are expressed by using the sub-unit time (1/N sec) and y.
The coded image signal 200a includes a header H containing sub-unit time data Dk that indicates N (natural number) for obtaining the sub-unit time, and the header H is followed by code sequences Sbn (Sb0, Sb1, Sb2, . . . ) corresponding to the respective frames F(n) (F(0), F(1), F(2), . . . ). Each code sequence Sbn contains display cycle multiplier data Dyn (Dy0, Dy1, Dy2, . . . ) indicating a display time h(n) (h(0), h(1), h(2), . . . ) which is measured by using the sub-unit time (1/N), and the number y of (1/N), with the time X as a reference.
In FIG. 17(c), Cgn (Cg0, Cg1, Cg2 . . . .) are coded image data corresponding to the respective frames F(n) (F(0), F(1), F(2), . . . ).
However, when the image VOP0 is an I-VOP (I frame), the VOP2 and VOP3 are B-VOPs (B frames), and the VOP4 is a P-VOP (P frame) as shown in FIG. 17(b), in the bit stream of the coded image signal 200a shown in FIG. 17(c), the P-VOP (VOP3) and the B-VOP (VOP1) are arranged as the code sequences of the frames F(1) and F(2) which follow the code sequence of the frame F(0) corresponding to the I-VOP (VOP0).
A description is now given of the drawbacks of the image signal data structures described with respect to FIGS. 14xcx9c16.
As described above, in a coded image signal obtained by coding an image signal having a fixed interval T of frame display timings, the image display timing h(n) of each frame is expressed by h(n)=nxe2x80x2xc3x97T, wherein nxe2x80x2 is the number indicating the order of display, and nxe2x80x2=B(n).
In other words, when the coded image signal having the fixed frame-display interval T (i.e., a coded signal of an image having a fixed frame rate) is decoded for display, if the period T (the fixed display interval) is detectable at the decoding end, the display time h(n) of the n-th frame F(n) in the transmission order can be uniquely decided by increasing the display interval T by nxe2x80x2 (=B(n)) times. Nevertheless, when decoding the coded image signal, there is no choice but to perform complicated display using the display time data Dtn (Dt0, Dt1, Dt2, . . . ) inserted in the coded image signals corresponding to the respective frames F(n) (F(0), F(1), F(2), . . . ) as shown in FIG. 14.
Next, a description is given of the drawbacks of the image signal data structures described with respect to FIGS. 17(a)xcx9c17(c).
As described above, in the image signal data structure proposed by the current MPEG4, even when the frame rate is fixed, the value of the frame rate cannot be known unless several frames are decoded and, therefore, it is difficult to simplify the circuit structure for implementing the actual decoding process.
This problem will be briefly described. When the VOP0 is an I-VOP (I frame), the VOP1 and the VOP2 are B-VOPs (B frames), and the VOP3 xe2x80x2is a P-VOP (P frame) as shown in FIG. 17(b), since the frame F(0) corresponding to the I-VOP (I frame) is followed by the frame F(1) corresponding to the P-VOP (P frame) and the frame F(2) corresponding to the B-VOP (B frame) in the bit stream of the coded image signal 200a shown in FIG. 17(c), the frame display cycle (1 fixed VOP increment), i.e., the interval between the display timing of the I-VOP and the display timing of the following B-VOP (B frame), cannot be known until the frame F(2) corresponding to the B-VOP (B frame) is transmitted.
The present invention is made to solve the above-described problems and has for its object to provide an image signal data structure which enables a reproduction process including decoding and image display for a coded image signal having a fixed cycle of reproduction for each frame at the decoding end, such as a fixed frame rate (cycle of image display for each frame) by using a simple hardware structure, and also enables a reproduction process for a coded image signal having a variable cycle of reproduction for each frame, such as a variable frame rate.
Another object of the present invention is to provide an image coding method and an image coding apparatus capable of performing an image coding process which enables a reproduction process including decoding and image display for a coded image signal having a fixed cycle of reproduction for each frame at the decoding end, such as a fixed frame rate (cycle of image display for each frame) by using a simple hardware structure, and also enables a reproduction process for a coded image signal having a variable cycle of reproduction for each frame, such as a variable frame rate.
Still another object of the present invention is to provide an image decoding method and an image decoding apparatus capable of performing an accurate reproduction process including decoding and image display for a coded image signal, according to whether the cycle of reproduction for each frame is variable or not.
Yet another object of the present invention is to provide an image signal data structure which enables detection of the value of a frame rate or the like of a coded image signal having a fixed frame rate or the like before decoding each frame, and simplifies various hardware structures for implementing a reproduction process including decoding and image display.
A further object of the present invention is to provide an image coding method and an image coding apparatus capable of detecting the value of a frame rate or the like of a coded image signal having a fixed frame rate or the like before decoding each frame, and performing a reproduction process including decoding and image display for the coded image signal by a simple hardware structure.
A still further object of the present invention is to provide an image decoding method and an image decoding apparatus capable of accurately decoding the coded image signal obtained by the above-described coding process.
Another object of the present invention is to provide a data storage medium containing a coded image signal having the above-described data structure, and a data storage medium containing an image processing program for implementing, with a computer, the above-described image coding method and image decoding method.
Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the spirit and scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a first aspect of the present invention, there is provided a data structure of an image signal for which image reproduction of each frame is performed at a prescribed cycle, and the image signal data structure includes a reproduction cycle identifier indicating whether the cycle of image reproduction for each frame is variable or not. Therefore, when the reproduction cycle for each frame is fixed, reproduction of the image signal, such as decoding of the coded image signal and display of the decoded image signal, can be performed by a simple circuit structure.
According to a second aspect of the present invention, in the image signal data structure of the first aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not. Therefore, when the image display cycle for each frame is fixed, display of decoded data obtained by decoding the coded image signal can be performed by a simple circuit structure.
According to a third aspect of the present invention, the image signal data structure of the second aspect includes a fixed display cycle identifier indicating that the cycle of image display for each frame is fixed, as the display cycle identifier; display cycle data indicating the cycle of image display for each frame; and frame position data corresponding to each frame and indicating the positional relationship of each frame with the previous and subsequent frames. Therefore, when the image display cycle for each frame is fixed, the timing of image display of the frame can be defined according to the display cycle data and the frame position data.
According to a fourth aspect of the present invention, the image signal data structure of the second aspect includes a variable display cycle identifier indicating that the cycle of image display for each frame is variable, as the display cycle identifier; and display timing data indicating the timing at which image display of each frame is performed, the timing being set relatively to a desired reference time selected from at least one reference time according to each frame. Therefore, when the image display cycle for each frame is variable, the timing of image display of the frame can be defined according to the display timing data.
According to a fifth aspect of the present invention, in the image signal data structure of the first aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding of a coded image signal corresponding to each frame is variable or not. Therefore, when the decoding cycle for each frame is fixed, decoding of the coded image signal can be performed by a simple circuit structure.
According to a sixth aspect of the present invention, the image signal data structure of the fifth aspect includes a fixed decoding cycle identifier indicating that the cycle of decoding for each frame is fixed, as the decoding cycle identifier; decoding cycle data indicating the cycle of decoding for each frame; and frame position data corresponding to each frame and indicating the positional relationship of each frame with the previous and subsequent frames. Therefore, when the decoding cycle for each frame is fixed, the timing of decoding of the frame can be defined according to the decoding cycle data and the frame position data.
According to a seventh aspect of the present invention, the image signal data structure of the fifth aspect includes a variable decoding cycle identifier indicating that the cycle of decoding for each frame is variable, as the decoding cycle identifier; and decoding timing data indicating the timing at which decoding of each frame is performed, the timing being set relatively to a desired reference time selected from at least one reference time, according to each frame. Therefore, when the decoding cycle for each frame is variable, the timing of decoding of the frame can be defined according to the decoding timing data.
According to an eighth aspect of the present invention, there is provided an image coding method for coding an image signal corresponding to a specific image to generate a coded image signal, and outputting the coded image signal together with a reproduction cycle identifier indicating whether the cycle of image reproduction for each of frames constituting the image is variable or not. In this method, when an image signal having a fixed cycle of image reproduction for each frame is input as the image signal to be coded, reproduction cycle data and frame position data are output together with the reproduction cycle identifier, the reproduction cycle data indicating the cycle of image reproduction for each frame, and the frame position data corresponding to each frame and indicating the positional relationship of each frame with the previous and subsequent frames. When an image signal having a variable cycle of image reproduction for each frame is input as the image signal to be coded, reproduction timing data is output together with the reproduction cycle identifier, the reproduction timing data indicating the timing at which image reproduction of each frame is performed, the timing being set relatively to a desired reference time selected from at least one reference time, according to each frame. In this method, an image signal having a variable cycle of image reproduction for each frame and an image signal having a fixed cycle of image reproduction for each frame can be coded together with data indicating the reproduction timing of each frame according to each image signal. Therefore, when the reproduction cycle for each frame is fixed, the bit number required for setting the reproduction time can be reduced and, furthermore, reproduction of the image having a variable frame reproduction cycle can be performed in the same manner as the conventional reproduction.
According to a ninth aspect of the present invention, in the image coding method of the eighth aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not; the reproduction cycle data is display cycle data indicating the cycle of image display for each frame; and the reproduction timing data is display timing data indicating the timing at which image display of each frame is performed, the timing being set relatively to a desired reference time selected from at least one reference time, according to each frame. Therefore, when the display cycle for each frame is fixed, the bit number required for setting the display time can be reduced and, furthermore, display of the image having a variable frame display cycle can be performed in the same manner as the conventional display.
According to a tenth aspect of the present invention, in the image coding method of the eighth aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding of coded image data corresponding to each frame is variable or not; the reproduction cycle data is decoding cycle data indicating the cycle of decoding for each frame; and the reproduction timing data is decoding timing data indicating the timing at which decoding of each frame is performed, the timing being set relatively to a desired reference time selected from at least one reference time, according to each frame. Thereby, when the decoding cycle for each frame is fixed, the bit number required for setting the decoding time of each frame can be reduced and, furthermore, decoding of the image having a variable frame decoding cycle can be performed in the same manner as the conventional decoding.
According to an eleventh aspect of the present invention, there is provided an image decoding method for decoding a coded image signal which includes coded image data corresponding to frames constituting an image, and a reproduction cycle identifier indicating whether the cycle of image reproduction for each frame is variable or not. In this method, when the reproduction cycle identifier indicates that the cycle of image reproduction for each frame is fixed, decoded image data obtained by decoding the coded image data corresponding to each frame becomes image data having a reproduction timing which is decided according to reproduction cycle data indicating the cycle of image reproduction for each frame and frame position data indicating the positional relationship of each frame with the previous and subsequent frames, which data are included in the coded image signal. When the reproduction cycle identifier indicates that the cycle of image reproduction for each frame is variable, decoded image data obtained by decoding the coded image data corresponding to each frame is image data having a reproduction timing decided according to reproduction timing data which is included in the coded image signal and indicates the timing at which image reproduction of each frame is carried out, the timing being set relatively to a desired reference time selected from at least one reference time, according to each frame. Therefore, reproduction of the coded image signal, including decoding and display, can be accurately performed according to whether the reproduction timing of each frame of the coded image signal is variable or not.
According to a twelfth aspect of the present invention, in the image decoding method of the eleventh aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not; the reproduction cycle data is display cycle data indicating the cycle of image display for each frame; and the reproduction timing data is display timing data indicating the timing at which image display of each frame is performed. Therefore, display of the image signal can be accurately performed according to whether the display timing of each frame of the coded image signal is variable or not.
According to a thirteenth aspect of the present invention, in the image decoding method of the eleventh aspect, a decoding timing at which decoding of each frame is carried out is set according to display timing data of plural frames including an object frame to be decoded. Therefore, not only display but also decoding of each frame can be carried out according to the display timing data.
According to a fourteenth aspect of the present invention, in the image decoding method of the thirteenth aspect, based on the display timing data of the object frame and the display timing data of the next frame transmitted subsequently to the object frame, the decoding timing of the object frame is set at a timing that is earlier, by a prescribed offset time, than the earlier display timing between the display timings of the object frame and the next frame. Therefore, decoding of each frame can be performed according to the display timing data, without impeding the flow of display.
According to a fifteenth aspect of the present invention, in the image decoding method of the fourteenth aspect, when the display timing of the object frame is earlier than the display timing of the next frame, the offset time is set to a length longer than the time required for decoding the object frame; and when the display timing of the next frame is earlier than the display timing of the object frame, the offset time is set to a length longer than the sum of the time required for decoding the object frame and the time required for decoding the next frame. Therefore, decoding of a coded image data corresponding to a series of frames including P-VOP and B-VOP as well as I-VOP can be performed according to the display timing data.
According to a sixteenth aspect of the present invention, in the image decoding method of the eleventh aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding of coded image data corresponding to each frame is variable or not; the reproduction cycle data is decoding cycle data indicating the cycle of decoding for each frame; and the reproduction timing data is decoding timing data indicating the timing at which decoding of each frame is performed. Therefore, decoding of the coded image signal can be accurately performed according to whether the decoding timing of each frame of the coded image signal is variable or not.
According to a seventeenth aspect of the present invention, the image signal data structure of the first aspect further includes sub-unit time data indicating the length of a sub-unit time which is obtained by dividing a prescribed time interval into N (natural number), by the natural number N; and reproduction cycle multiplier data indicating that the cycle of image reproduction for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, when a coded image signal having a fixed frame rate is processed, the value of the frame rate can be detected before decoding each frame, whereby various hardware structures for implementing reproduction including decoding and display can be simplified.
According to an eighteenth aspect of the present invention, in the image signal data structure of the seventeenth aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not; and the reproduction cycle multiplier data is display cycle multiplier data which indicates that the cycle of image display for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, various hardware structures for implementing image display can be simplified.
According to a nineteenth aspect of the present invention, in the image signal data structure of the seventeenth aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding for each frame is variable or not; and the reproduction cycle multiplier data is decoding cycle multiplier data which indicates that the cycle of decoding for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, various hardware structures for implementing decoding can be simplified.
According to a twentieth aspect of the present invention, there is provided an image coding method for coding an image signal corresponding to a specific image to generate a coded image signal, and outputting the coded image signal together with a reproduction cycle identifier indicating whether the cycle of image reproduction for each of frames constituting the image is variable or not. In this method, when an image signal having a fixed cycle of image reproduction for each frame is input as the image signal to be coded, sub-unit time data and reproduction cycle multiplier data are output together with the reproduction cycle identifier, the sub-unit time data indicating the length of a sub-unit time which is obtained by dividing a prescribed time interval into N (natural number), by the natural number N, and the reproduction cycle multiplier data indicating that the cycle of image reproduction for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, when a coded image signal having a fixed frame rate is processed, the value of the frame rate can be detected before decoding each frame, whereby reproduction including decoding and display can be performed with a simple hardware structure.
According to a twenty-first aspect of the present invention, in the image coding method of the twentieth aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not; and the reproduction cycle multiplier data is display cycle multiplier data which indicates that the cycle of image display for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, various hardware structures for implementing display at the decoding end can be simplified.
According to a twenty-second aspect of the present invention, in the image coding method of the twentieth aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding for each frame is variable or not; and the reproduction cycle multiplier data is decoding cycle multiplier data which indicates that the cycle of decoding for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, various hardware structures for implementing decoding can be simplified.
According to a twenty-third aspect of the present invention, there is provided an image decoding method for decoding a coded image signal including the following data: coded image data corresponding to frames constituting an image; a reproduction cycle identifier indicating whether the cycle of image reproduction for each frame is variable or not; sub-unit time data indicating the length of a sub-unit time which is obtained by dividing a prescribed time interval into N (natural number), by the natural number N; and reproduction cycle multiplier data indicating that the cycle of image reproduction for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. In this method, when the reproduction cycle identifier indicates that the cycle of image reproduction for each frame is fixed, decoded image data obtained by decoding the coded image data corresponding to each frame makes image data having a reproduction timing which is decided according to the sub-unit time data indicating the length of the sub-unit time and the reproduction cycle multiplier data indicating the cycle of image reproduction for each frame. Therefore, when a coded image signal having a fixed frame rate is processed, the value of the frame rate can be detected before decoding each frame, whereby the hardware structure at the decoding end can be simplified.
According to a twenty-fourth aspect of the present invention, in the image decoding method of the twenty-third aspect, the reproduction cycle identifier is a display cycle identifier indicating whether the cycle of image display for each frame is variable or not; and the reproduction cycle multiplier data is display cycle multiplier data which indicates that the cycle of image display for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, the hardware structure for display at the decoding end can be simplified.
According to a twenty-fifth aspect of the present invention, in the image decoding method of the twenty-third aspect, the reproduction cycle identifier is a decoding cycle identifier indicating whether the cycle of decoding for each frame is variable or not; and the reproduction cycle multiplier data is decoding cycle multiplier data which indicates that the cycle of decoding for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, the hardware structure for decoding at the decoding end can be simplified.
According to a twenty-sixth aspect of the present invention, there is provided an image coding apparatus for coding an image signal corresponding to a specific image, and the apparatus comprises an encoder for coding an input image signal and outputting coded image data; a cycle decision unit for deciding whether the cycle of image reproduction for each of frames constituting the image is variable or not, based on the image signal, and outputting a reproduction cycle identifier indicating the result of the decision; a first data generator for generating reproduction cycle data indicating the cycle of image reproduction for each frame, according to the image signal; a second data generator for generating frame position data corresponding to each frame and indicating the positional relationship of each frame with the previous and subsequent frames, according to the image signal; a third data generator for generating reproduction timing data indicating the timing at which image reproduction for each frame is carried out, according to the image signal; an ON/OFF switch for switching the circuit between the ON state where the reproduction cycle data is transmitted and the OFF state where the reproduction cycle data is cut off, according to the reproduction cycle identifier; a selector switch for selecting one of the frame position data and the reproduction timing data, according to the reproduction cycle identifier; a multiplexer for multiplexing the outputs from the encoder, the cycle decision unit, and the switches in a prescribed order; and the image coding apparatus outputting a bit stream obtained by the multiplication as a coded image signal. In this apparatus, an image signal having a variable cycle of image reproduction for each frame and an image signal having a fixed cycle of image reproduction for each frame can be coded with data indicating the reproduction timing of each frame according to each image signal. Therefore, when the reproduction cycle for each frame is fixed, the bit number required for setting the reproduction time can be reduced and, furthermore, reproduction of the image having a variable frame reproduction cycle can be performed easily.
According to a twenty-seventh aspect of the present invention, there is provided an image decoding apparatus for decoding and reproducing the coded image signal output from the image coding apparatus of the twenty-sixth aspect, and the apparatus comprises a demultiplexer for receiving the coded image signal, and separating the coded image signal into the coded image data, the reproduction cycle identifier, the reproduction cycle data, the frame position data, and the reproduction timing data, and then outputting these data; a decoder for decoding the coded image data, frame by frame, to generate decoded image data; an ON/OFF switch for switching the circuit between the ON state where the reproduction cycle data is transmitted and the OFF state where the reproduction cycle data is cut off, according to the reproduction cycle identifier; a selector switch for selecting one of the frame position data and the reproduction timing data, according to the reproduction cycle identifier; and a display unit for performing image display of each frame according to the decoded image data; wherein at least one of the decoding by the decoder and the image display by the display unit is performed at a reproduction timing decided by the reproduction cycle data and the frame position data or a reproduction timing decided by the reproduction timing data, according to the reproduction cycle identifier. Therefore, reproduction of the coded image signal, including decoding and display, can be accurately performed according to whether the reproduction timing of each frame of the coded image signal is variable or not.
According to a twenty-eighth aspect of the present invention, there is provided an image coding apparatus for coding an image signal corresponding to a specific image, and the apparatus comprises an encoder for coding an input image signal and outputting coded image data; a cycle decision unit for deciding whether the cycle of image reproduction for each of frames constituting the image is variable or not, based on the image signal, and outputting a reproduction cycle identifier indicating the result of the decision; a first data generator for generating sub-unit time data indicating the length of a sub-unit time which is obtained by dividing a prescribed time interval into N (natural number), by the natural number N; a second data generator for generating reproduction cycle multiplier data indicating that the cycle of image reproduction for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M, according to the image signal; a third data generator for generating reproduction timing data indicating the timing at which image reproduction of each frame is carried out, according to the image signal; a first ON/OFF switch for switching the circuit between the ON state where the sub-unit time data is transmitted and the OFF state where the sub-unit time data is cut off, according to the reproduction cycle identifier; a second ON/OFF switch for switching the circuit between the ON state where the reproduction cycle multiplier data is transmitted and the OFF state where the reproduction cycle multiplier data is cut off, according to the reproduction cycle identifier; a multiplexer for multiplexing the outputs from the encoder, the cycle decision unit, the third data generator, and the first and second ON/OFF switches in a prescribed order; and the image coding apparatus outputting a bit stream obtained by the multiplication as a coded image signal. Therefore, when a coded image signal having a fixed frame rate is processed at the decoding end, the value of the frame rate can be detected before decoding each frame, whereby reproduction of the coded image signal, including decoding and display, can be performed by a simple hardware structure.
According to a twenty-ninth aspect of the present invention, there is provided an image decoding apparatus for decoding and reproducing the coded image signal output from the image coding apparatus of the twenty-eighth aspect, and the apparatus comprises a demultiplexer for receiving the coded image signal, and separating the coded image signal into the coded image data, the reproduction cycle identifier, the sub-unit time data, the reproduction cycle multiplier data, and the reproduction timing data, and then outputting these data; a decoder for decoding the coded image data, frame by frame, to generate decoded image data; a first ON/OFF switch for switching the circuit between the ON state where the reproduction cycle multiplier data is transmitted and the OFF state where this data is cut off, according to the reproduction cycle identifier; a second ON/OFF switch for switching the circuit between the ON state where the reproduction timing data is transmitted and the OFF state where this data is cut off, according to the reproduction cycle identifier; a display unit for performing image display of each frame according to the decoded image data; wherein at least one of the decoding by the decoder and the image display by the display unit is performed at a reproduction timing decided by the sub-unit time data and the reproduction cycle multiplier data or a reproduction timing decided by the reproduction timing data, according to the reproduction cycle identifier. Therefore, when a coded image signal having a fixed frame rate is processed, the value of the frame rate can be detected before decoding each frame, whereby the hardware structure at the decoding end can be simplified.
According to a thirtieth aspect of the present invention, there is provided a data storage medium containing an image signal for which image reproduction for each frame is performed at a prescribed cycle, and the image signal includes a reproduction cycle identifier indicating whether the cycle of image reproduction for each frame is variable or not. Therefore, by using the data storage medium, when the reproduction cycle for each frame is fixed, reproduction of the image signal, such as decoding of the coded image signal and display of the decoded image signal, can be performed with a simple circuit structure.
According to a thirty-first aspect of the present invention, there is provided a data storage medium containing an image signal for which image reproduction for each frame is performed at a prescribed cycle, and the image signal includes a reproduction cycle identifier indicating whether image reproduction for each frame is variable or not; sub-unit time data indicating the length of a sub-unit time which is obtained by dividing a prescribed time interval into N (natural number), by the natural number N; and reproduction cycle multiplier data indicating that the cycle of image reproduction for each frame is the sub-unit time multiplied by M (natural number), by the multiplier M. Therefore, by using the data storage medium, the value of a fixed frame rate of a coded image signal can be detected before decoding each frame, whereby various hardware structures for implementing reproduction including decoding and display can be simplified.
According to a thirty-second aspect of the present invention, there is provided a data storage medium containing an image processing program, and the image processing program is a coding program which enables a computer to execute the coding of an image signal by the image coding method of the eighth aspect. Therefore, by loading the program in a computer, it is possible to implement an apparatus that can reduce the bit number required for setting the reproduction time when the frame reproduction cycle is fixed, and that can facilitate reproduction of an image having a variable frame reproduction cycle.
According to a thirty-third aspect of the present invention, there is provided a data storage medium containing an image processing program, and the image processing program is a decoding program which enables a computer to execute the decoding of a coded image signal by the image decoding method of the eleventh aspect. Therefore, by loading the program in a computer, it is possible to implement an apparatus that can accurately perform reproduction of the coded image signal, including decoding and display, according to whether the reproduction timing of each frame is variable or not.
According to a thirty-fourth aspect of the present invention, there is provided a data storage medium containing an image processing program, and the image processing program is a coding program which enables a computer to execute the coding of an image signal by the image coding method of the twentieth aspect. Therefore, by loading the program in a computer, it is possible to implement an apparatus that can detect the value of a fixed frame rate of a coded image signal before decoding each frame, and that can perform reproduction of the coded image signal, including decoding and display, by a simple hardware structure.
According to a thirty-fifth aspect of the present invention, there is provided a data storage medium containing an image processing program, and the image processing program is a decoding program which enables a computer to execute the decoding of a coded image signal by the image decoding method of the twenty-third aspect. Therefore, by loading the program in a computer, it is possible to implement an apparatus that can detect the value of a fixed frame rate of a coded image signal before decoding each frame, and that can simplify the hardware structure at the decoding end.